This invention relates generally to shared data stores, and more particularly to registering and using data conversion algorithms such as for compression/decompression or encryption/decryption of data in large shared data stores, such as data warehouses.
It is known in the data storage industry to compress data in order to conserve storage space on disk drives, particularly large data stores, and to reduce access time. Most data stores (databases) have a several different built-in compression algorithms which may be selected by users to compress data. These compression algorithms are typically generic, well known, general-purpose compression algorithms that are capable of doing a reasonable job of compressing a variety of different types of data. Some of these general-purpose compression algorithms are better suited for compressing certain types of data such as, for example, numerical data or textual data, than other types of data, which is why several different compression algorithms may be made available to users. Large data stores, however, generally store many different types of data which may have very different compression needs. General-purpose compression algorithms may not be well suited to some of these compression needs, or to the structure of the underlying data or to the data domain. Accordingly, they may not be very time efficient space-wise or time-wise, which affects database performance. Further, they may not provide the level or the quality of the compression desired for different types of data and different data domains, and may actually degrade the data.
Although it is possible for users to have custom special-purpose user-defined compression algorithms incorporated into their data stores to meet particular needs, with presently available databases providing custom built-in compression algorithms requires modification of the database code which is a complex task that requires skilled programmers. Moreover, this is a rather inflexible approach. Compression algorithms are not easily modified or replaced once they have been incorporated into a database. As new data types are added or as compression requirements change, compression algorithms must also change. The installed compression algorithms may not be adaptable or able to accommodate the changes, and different compression schemes may be required. Additionally, users may wish to apply different types of compression to different data types and to different data organizations, such as in rows or columns of a table. To enable this it is necessary to incorporate the different compression algorithms into the data store which, as discussed, is a difficult and complex task with currently available databases.
There is a need for approaches that allow users of shared data stores the ability to incorporate easily a variety of arbitrary custom or special-purpose data conversion schemes such as compression schemes into large data stores and to apply selectively the different data conversion schemes to data. There is also a need as well as to change or modify available data conversion schemes as needs change. It is desirable to provide systems and methods that address these and other needs and problems associated with presently available data store data conversion approaches, and it is to these ends that the present invention is directed.